Many low-cost sensors (or cameras) may spatially or electronically undersample an image. Similarly, cameras taking pictures from great distances, such as aerial photos, may not obtain detailed information about the subject matter. This may result in aliased images in which the high frequency components are folded into the low frequency components in the image. Consequently, subtle or detail information (high frequency components) are not present in the images. Super-resolution image reconstruction generally increases image resolution without necessitating a change in the design of the optics and/or detectors by using a sequence (or a few snapshots) of low-resolution images. Super-resolution image reconstruction algorithms effectively de-alias undersampled images to obtain a substantially alias-free or, as identified in the literature, a super-resolved image.
When undersampled images have sub-pixel translations and rotations between successive frames, they contain different information regarding the same scene. Super-resolution image reconstruction involves, inter alia, combining information contained in undersampled images to obtain an alias-free (high-resolution) image. Super-resolution image reconstruction from multiple snapshots, taken by a detector which has shifted and/or rotated in position, provides far more detail information than any interpolated image from a single snapshot.
Generally, super-resolution image reconstruction involves acquiring a sequence of images from the same scene with sub-pixel translations and rotations among the images. The methods for acquiring low-resolution images with sub-pixel translations and rotations include acquiring images affected by translation and/or rotation.
One of these methods is to have controlled sub-pixel translations and rotations in which each image is captured by linearly displacing the sensor a known fraction pixel displacement, as for example in U.S. Pat. No. 6,344,893, “Super-resolving imaging system,” Feb. 5, 2002, D. Mendlovic et. al.; and U.S. Pat. No. 6,483,952, “Super resolution methods for electro-optical systems,” Nov. 19, 2002, D. D. Gregory, et. al. Or the super-resolution image reconstruction may involve images acquired by rotating an image array about the optical axis as, for example, in U.S. Pat. No. 6,642,497, “System for improving image resolution via sensor rotation,” Nov. 4, 2003, J. Apostolopoulos, et al. Generally, the acquired images in super-resolution image reconstruction contain sub-pixel translations and/or rotations.
Methods for super-resolving images that contain sub-pixel translations and/or rotations are in the following references.
U.S. Pat. No. 7,323,681, to Oldham, et. al., “Image enhancement by sub-pixel imaging,” issued Jan. 29, 2008, teaches a method of using a linear reconstruction approach as described in A. S. Fruchter, et al., “Drizzle: a Method for the Linear Reconstruction of Undersampled Images,” Publications of the Astronomical Society of the Pacific, vol. 114, pp. 144-152, February 2002. In this method, the target's orientation relative to the detector may be rotated and/or translated so as to yield a generalized mapping between a pixel space of the low-resolution image and a relatively smaller pixel space of the high-resolution image. The sub-pixel's output, representing high-resolution image pixel values, may be achieved by summing the contributions of image values of the overlapping pixels from a number of low-resolution images. In order to improve sub-pixel output approximation, the mapping includes a surface intensity term which compensates for the small overlapping of pixels due to rotation.
U.S. Pat. No. 7,106,914, “Bayesian image super resolution,” Sep. 12, 2006, M. E. Tipping, et. al. and U.S. Patent Application Publication US2007/0130095, “Bayesian approach for sensor super-resolution,” Jun. 7, 2007, M. A Peot and M Aguilar disclose a statistical Bayesian inverse processing method in which an observation model is formulated to relate the original high-resolution image to the observed low-resolution images. Parameters including rotation and translation are obtained by optimizing a marginal likelihood of low-resolution images in which the marginal likelihood is a function of the alignment and rotation parameters. Then, the full high-resolution image is derived in an inference operation using these parameters.
U.S. Patent Application Publication US2007/0196009, “Method for operating an X-ray diagnostic device for the generation of high-resolution images,” Aug. 23, 2007, to Deinzer discloses a method involving translations and rotations among low-resolution X-ray images that are determined by a minimization approach. The high-resolution X-ray image is obtained by performing bilinear interpolation approach.
There is a need to produce high-resolution images from a sequence of low-resolution images captured by a low-cost imaging device that has experienced translations and/or rotations. The amount of sub-pixel translation and/or rotation may be unknown and it creates a need to estimate sub-pixel translation and rotation. In order to produce high-resolution images from a sequence of translated and rotated undersampled (low-resolution) images, there exists a need to eliminate aliasing, while taking into account natural jitter.